Skip to content

ci(eql): port and harden the inert EQL release pipeline - #941

Merged
tobyhede merged 5 commits into
mainfrom
toby/cip-3742-eql-release-pipeline
Aug 25, 2026
Merged

ci(eql): port and harden the inert EQL release pipeline#941
tobyhede merged 5 commits into
mainfrom
toby/cip-3742-eql-release-pipeline

Conversation

@tobyhede

@tobyhede tobyhede commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Summary

EQL is the PostgreSQL layer that stores and queries encrypted data. Its subtree import brought eleven files under packages/eql/.github/, including the release pipeline, but GitHub only reads workflows from the repository root. The npm package, Rust crate, SQL bundle, docs bundle, and Postgres image therefore had no executable release path in this repository.

This PR ports and hardens the viable release pipeline at the repository root while keeping it inert. It does not arm EQL publishing.

Inertness is derived from FROZEN_PUBLISHERS in scripts/release-gate.mjs, not from a second release flag:

  • scripts/eql-pipeline-armed.mjs gates the EQL SQL, docs, crate, image, and prerelease npm jobs.
  • Production npm publishing is held by release-gate.mjs, which fails the shared release gate while EQL remains frozen.

Both paths read the same map entry. Removing @cipherstash/eql from that map during cutover arms the complete pipeline; there is no independent switch to forget.

Changes

Release workflows

Upstream file Result here
_build-sql.yml .github/workflows/_build-eql-sql.yml
_build-docs.yml .github/workflows/_build-eql-docs.yml
release.yml merged into .github/workflows/release.yml because npm trusted publishing binds to the workflow filename
release-plz.yml moved to the repository root without renaming because crates.io trusted publishing binds to it
release-postgres-eql-image.yml moved to the repository root
lint-release.yml merged into the existing root workflow
rebuild-docs.yml deliberately retired; it targeted the old docs site through the deprecated DOCS_WEBHOOK_URL
actionlint.yaml, ISSUE_TEMPLATE/docs-feedback.yml dropped because equivalent root configuration already exists
.github/release.yml dropped because this release path does not use GitHub-generated release notes
workflows/README.md dropped because it documented an upstream merge-queue model this repository does not use

The imported packages/eql/.github/ directory is deleted. scripts/__tests__/eql-suite-ci.test.mjs asserts that it stays absent and that no root workflow consumes the retired docs webhook secret.

Monorepo path corrections

  • The npm manifest lives at packages/eql/packages/eql, two levels below the subtree root.
  • Every EQL mise invocation runs from packages/eql, where mise can discover the subtree configuration.
  • release-plz/action receives explicit manifest_path and config inputs; composite-action steps do not inherit defaults.run.working-directory.
  • Prerelease builds run through Turbo so dependency builds are included.
  • Artifact uploads fail when expected SQL or docs files are absent.

Release correctness

  • Version classification requires an exact X.Y.Z-(alpha|beta|rc).N prerelease identity before any tag or release is created.
  • The SQL release workflow runs mise run --force build --version .... --version is not a declared mise source, so an unforced build can reuse SQL stamped for an earlier release when the SQL sources are unchanged.
  • A regression test pins that forced invocation.
  • verify-release-assets.mjs validates the SQL schema-version stamp against the package version. Its former eql_v3 substring check also passed placeholder bundles.
  • Prereleases no longer move npm's latest tag after GA.
  • The EQL run summary includes the shared release gate and production release job, so an inert run is distinguishable from a failed publish.

Supply-chain hardening

  • Remote actions are SHA-pinned with version comments.
  • Release workflows explicitly disable supported GitHub Actions cache paths.
  • Docker setup cache defaults are disabled, and Docker build cache inputs are forbidden on publishing paths.
  • Publishing permission checks distinguish jobs allowed to write repository state from the smaller set allowed to request OIDC tokens.
  • release-plz.yml runs only for EQL paths and checks crates.io before importing the signing key.
  • The Postgres image source label points at cipherstash/stack, preventing a future push from relinking the GHCR package to the old repository.

Guards and audit fixes

  • EQL and protect-ffi Cargo workspaces are both covered by crate publish opt-out checks.
  • Published README discovery is derived from pnpm-workspace.yaml, including nested EQL and per-platform FFI packages.
  • Repository scans skip local .claude/worktrees checkouts.
  • pnpm dev reaches nested workspace packages with ./packages/**.
  • EQL dependency pins are checked across Cargo manifests, Cargo configuration, package manifests, and pnpm workspace configuration.
  • Documentation describes durable release mechanisms instead of asserting mutable registry configuration.

Verification

  • pnpm run test:scripts — 49 files, 816 passed, 1 skipped.
  • Biome — 866 tracked files checked with zero errors.
  • actionlint passes for the changed release workflows.
  • scripts/lint-no-workflow-caching.mjs exits 0.
  • scripts/lint-no-eql-registry-pins.mjs exits 0.
  • git diff --check passes.

A temporary EQL patch bump was used to exercise the lockstep version hook. It moved the npm package, eql-bindings crate, SQL schema stamps, release manifests, and generated TypeScript manifest to one version. In that bumped state, release-gate.mjs refused publishing while EQL remained frozen.

The publish-time SQL verifier was also driven against a DEV-stamped bundle with an otherwise agreeing manifest: it exits non-zero and names the incorrect schema stamp.

The docs workflow still needs its first execution in this repository when the pipeline is armed. It builds and attaches the versioned HTML/XML/Markdown artifact; there is intentionally no external docs-site webhook.

Cutover notes

This PR builds the pipeline but does not perform the cutover. Before removing the EQL entry from FROZEN_PUBLISHERS, verify the external publisher configuration required by npm and crates.io and the GHCR release path. The postgres-eql image is also currently a major version behind EQL, so moving its floating tags should be deliberate.

Related

Refs #885

Review notes

Start with scripts/eql-pipeline-armed.mjs and scripts/release-gate.mjs, then review the production and prerelease EQL paths in .github/workflows/release.yml.

workflow_dispatch is the notable behavior change to the existing shared release workflow. Production jobs are explicitly gated so a dispatch from a feature branch cannot publish JS or FFI packages.

No changeset: this PR changes repository tooling, tests, and documentation. The modified EQL prepublishOnly verifier is excluded from the published package by its files allowlist.

The EQL subtree arrived with eleven files under `packages/eql/.github/`, a
directory GitHub never reads. Seven of them were workflows, and between them
they are the entire publishing pipeline for `@cipherstash/eql`: the npm
package, the `eql-bindings` crate, the SQL bundle, the docs bundle and the
`postgres-eql` image, all of which ship at one version. None of it executed.

This ports the lot and keeps it inert, so it can be reviewed and dry-run
before trusted publishing is repointed at this repository.

Inertness is a derived switch rather than a flag. `scripts/eql-pipeline-armed.mjs`
reads `FROZEN_PUBLISHERS` in `scripts/release-gate.mjs` — the map that already
records "this package lives here and is published elsewhere" — and every job
that publishes an EQL artefact is gated on its answer. The cutover has to delete
that entry (the release gate refuses every release until it does), so deleting
it is what arms the pipeline; there is no second flag to forget, and forgetting
one would fail silently in the direction that publishes an npm package with no
SQL release, no docs and no crate.

Three paths had to be rewritten because the subtree root is not the package
root here: the npm manifest is two levels down at `packages/eql/packages/eql`,
mise config is only found from `packages/eql`, and `release-plz/action` needs
explicit `manifest_path` / `config` inputs because a composite action's steps
do not inherit the caller's `defaults.run.working-directory`.

Three of this repo's guards had to grow rather than be exempted, and two real
defects came out of them: the prerelease publish job was missing `node-gyp`
before its install, and built with `pnpm --filter` rather than through turbo.

Claude-Session: https://claude.ai/code/session_01T26DhFPkLfN3qPFDq2ttKw
@changeset-bot

changeset-bot Bot commented Aug 24, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 37c3296

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Seven findings from the review of the ported pipeline, all confirmed before
acting. Three were guards that could not fire, which is the failure mode this
repository cares most about — a check that reads as protection and is incapable
of reporting anything.

`rebuild-docs.yml` was the worst of them, and worse than reported: it has never
run, not once, against `@cipherstash/stack@1.0.0`, `1.1.0` and `1.1.1`. A ref
created with GITHUB_TOKEN starts no workflow run, and changesets creates those
tags with that token — the same rule this pipeline already works around for the
image build. Porting EQL's `eql-*` tag trigger into it would have added a second
dead trigger. The EQL docs rebuild is now a job in `release.yml` that fires the
webhook from inside the publishing run; the `@cipherstash/stack@*` half is left
alone as a pre-existing bug, recorded in that file's header so the next person
does not repeat the mistake.

`release-plz.yml`'s "refusing to publish the DEV placeholder" guard grepped for
`eql_v3`, which appears 23,723 times in every build of that bundle including a
DEV one. It now matches the schema version stamp against the crate version —
the only line in the bundle that records which build produced it.

`classify` accepted any version containing a hyphen where
`prepare-bindings-assets.sh` requires `X.Y.Z-(alpha|beta|rc).N`. Under the loose
check, `3.0.6-beta` got a public tag and GitHub release before the npm job died
on it.

Also: `PRE_GA_LATEST` was still `true` in the EQL publish script, so the first
prerelease cut through the new job would have moved the `latest` dist-tag off
the GA release. The file's own comment said to flip it once 3.0.0 GA shipped;
npm's `latest` is 3.0.5.

Two documentation defects of my own (a paragraph duplicated into AGENTS.md, and
a SECURITY.md sentence contradicting the permissions split this PR introduces),
one stale plan checkbox, an over-deep checkout, and a module-scope `execFileSync`
whose failure took 40 unrelated assertions with it.

Comment volume across everything this branch added is cut by roughly half.

Claude-Session: https://claude.ai/code/session_01T26DhFPkLfN3qPFDq2ttKw
@tobyhede
tobyhede force-pushed the toby/cip-3742-eql-release-pipeline branch from 463f8b4 to ab4c4f3 Compare August 24, 2026 04:46
@tobyhede
tobyhede marked this pull request as ready for review August 25, 2026 00:53
@tobyhede
tobyhede requested a review from a team as a code owner August 25, 2026 00:53
@tobyhede
tobyhede requested a review from a team August 25, 2026 00:55
@tobyhede tobyhede changed the title ci(eql): port the EQL release pipeline to the repository root, inert ci(eql): port and harden the inert EQL release pipeline Aug 25, 2026
@tobyhede
tobyhede merged commit 6ed8a23 into main Aug 25, 2026
30 checks passed
@tobyhede
tobyhede deleted the toby/cip-3742-eql-release-pipeline branch August 25, 2026 04:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants